CLAIMS 

1. An image matching system for retrieving a reference image 
similar to an input image, the image matching system comprising: 

means for making a match between the input image and a plurality 
5 of representative three-dimensional object models; 

means for making a match between the reference image and the 
representative three-dimensional object models! and 

means for retrieving the reference image similar to the input image 
by using a result of the match between the input image and the 
10 representative three-dimensional object models and a result of the match 
between the reference image and the representative three-dimensional 
object models. 

2. The image matching system according to claim 1, further 
comprising: 

1 5 means for finding a reference three-dimensional object model 

associated with the reference image similar to the input image! and 

means for newly retrieving the reference image similar to the input 
image by using the reference three-dimensional object model and the input 
image. 

20 3. The image matching system according to claim 1, further 

comprising: 

means for finding a reference three-dimensional object model 
associated with the reference image similar to the input image; 

conversion means for squaring an input condition of the input image 
25 with that of the reference image by converting the input image and/or the 
reference image on the basis of the reference three-dimensional object 
model; and 
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means for retrieving the reference image associated with the input 
image by making a match between the input image and reference image 
squared in input condition. 

4. The image matching system according to claim 3, wherein the 

5 conversion means previously converts the reference image, and squares an 
input condition of the input image with that of the reference image. 

5. The image matching system according to claim 1, comprising- 
image input means for inputting the input image; 

a representative three-dimensional object model storage section for 
10 storing a plurality of representative three-dimensional object models; 

image generation means for generating at least one comparison 
image close in input condition to the input image every representative 
three-dimensional object model on the basis of the representative three- 
dimensional object models stored in the representative three-dimensional 
1 5 object model storage section; 

image matching means for calculating a similarity between the 
input image and each of the comparison images generated by the image 
generation means, selecting a maximum similarity with respect to* 
comparison images associated with each representative three-dimensional 
20 object model, and regarding the maximum similarity as a similarity 

between the input image and the representative three-dimensional object 
model; 

a reference image storage section for storing the reference images of 

objects; 

25 a reference image matching result storage section for storing 

similarities between the reference images stored in the reference image 
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storage section and representative three-dimensional object models stored 
in the representative three-dimensional object model storage section; and 

result matching means for extracting the reference image similar to 
the input image on the basis of similarities between the input image and the 
5 representative three-dimensional object models calculated by the image 
matching means and similarities between the reference images and the 
representative three-dimensional object models stored in the reference , 
image matching result storage section. 

6. The image matching system according to claim 5, further 
10 comprising- 

three-dimensional object model registration means for registering 
representative three-dimensional object models in the representative three- 
dimensional object model storage section; 

reference image registration means for registering reference images 

1 5 in the reference image storage section; and 

reference image matching result update means for conducting 
calculation of the similarities using the image matching means, on a 
combination of a reference image and a representative three-dimensional 
object model newly generated by registration, when a new representative 

20 three-dimensional object model is registered in the representative three- 
dimensional object model storage section by the three-dimensional object 
model registration means, or when a new reference image is registered in 
the reference image storage section by the reference image registration 
means, and adding a result of the calculation to results in the reference 

25 image matching result storage section. 

7. The image matching system according to claim 5, wherein 
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the image matching means calculates a similarity between the input 
image and a representative three-dimensional object model every partial 
region, 

the reference image matching result storage section stores 
5 similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
in the representative three-dimensional object model storage section, every 
partial region, and 

the result matching means extracts the reference image similar to 
1 0 the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated by the image 
matching means every partial region and similarities between the reference 
images and the representative three-dimensional object models, stored in 
the reference image matching result storage section every partial region. 
1 5 8. The image matching system according to claim 5, wherein 

the result matching means calculates similarities between 
similarities between the input image and the representative three- 
dimensional object models and similarities between the reference images 
and the representative three-dimensional object models, and in the 
20 calculation provides the resultant similarities with weights on the basis of 
candidate precedence of similarities between the input image and the 
comparison images and the representative three-dimensional object models. 
9. The image matching system according to claim 2, comprising* 
image input means for inputting the input image; 
25 a representative three-dimensional object model storage section for 

storing a plurality of representative three-dimensional object models! 
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image generation means for generating at least one comparison 
image close in input condition to the input image every representative 
three-dimensional object model on the basis of the representative three- 
dimensional object models stored in the representative three-dimensional 
5 object model storage section; 

image matching means for calculating a similarity between the 
input image and each of the comparison images generated by the image 
generation means, selecting a maximum similarity with respect to 
comparison images associated with each representative three-dimensional 
10 object model, and regarding the maximum similarity as a similarity 

between the input image and the representative three-dimensional object 
model; 

a reference image storage section for storing the reference images of 

objects; 

1 5 a reference image matching result storage section for storing 

similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
in the representative three-dimensional object model storage section; 

result matching means for extracting the reference image similar to 

20 the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated by the image 
matching means and similarities between the reference images and the 
representative three-dimensional object models stored in the reference 
image matching result storage section! 

25 a reference three-dimensional object model storage section for 

storing reference three-dimensional object models associated with the 
reference images stored in the reference image storage section; 
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second image generation means for obtaining reference three- 
dimensional object models associated with reference images extracted from 
the result matching means, from the reference three-dimensional object 
model storage section, and generating at least one second comparison image 
5 close in input condition to the input image every reference three- 
dimensional object model on the basis of the obtained reference three- 
dimensional object models; and 

second image matching means for calculating similarities between 
the input image and second comparison images generated by the second 
1 0 image generation means, selecting a maximum similarity from among 
second comparison images associated with each of the reference three- 
dimensional object models, and regarding the maximum similarity as a 
similarity between the input image and the reference three-dimensional 
object model. 

1 5 10. The image matching system according to claim 9, further 

comprising' 

three-dimensional object model registration means for registering 
representative three-dimensional object models in the representative three- 
dimensional object model storage section; 

20 reference image registration means for registering reference images 

in the reference image storage section; and 

reference image matching result update means for conducting 
calculation of the similarities using the image matching means, on a 
combination of a reference image and a representative three-dimensional 

25 object model newly generated by registration, when a new representative 
three-dimensional object model is registered in the representative three- 
dimensional object model storage section by the three-dimensional object 
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model registration means, or when a new reference image is registered in 
the reference image storage section by the reference image registration 
means, and adding a result of the calculation to results in the reference 
image matching result storage section; and 
5 three-dimensional object model generation means responsive to 

registration of a similarity between the reference image and the 
representative three-dimensional object model in the reference image 
matching result storage section conducted by the reference image matching 
result update means, for generating the reference three-dimensional object 

1 0 model associated with the reference image by combining the representative 
three-dimensional object models stored in the representative three- 
dimensional object model storage section on the basis of the similarity, and 
registering the generated reference three-dimensional object model in the 
reference three-dimensional object model storage section. 

15 11. The image matching system according to claim 10, wherein 

the three-dimensional object model generation means generates a 
reference three-dimensional object model associated with each reference 
image by combining representative three-dimensional object models stored 
in the representative three-dimensional object model storage section every 

20 partial region, on the basis of similarities obtained every partial region 

between reference images stored in the reference image storage section and 
representative three-dimensional object models stored in the representative 
three-dimensional object model storage section, and registers the generated 
reference three-dimensional object model in the reference three-dimensional 

25 object model storage section. 

12. The image matching system according to claim 9, wherein 
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the image matching means calculates a similarity between the input 
image and a representative three-dimensional object model every partial 
region, 

the reference image matching result storage section stores 
5 similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
in the representative three-dimensional object model storage section, every 
partial region, and 

the result matching means extracts the reference image similar to 
10 the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated by the image 
matching means every partial region and similarities between the reference 
images and the representative three-dimensional object models, stored in 
the reference image matching result storage section every partial region. 
15 13. The image matching system according to claim 9, wherein 

the result matching means calculates similarities between 
similarities between the input image and the representative three- 
dimensional object models and similarities between the reference images 
and the representative three-dimensional object models, and in the 
20 calculation, provides the resultant similarities with weights on the basis of 
candidate precedence of similarities between the input image and the 
comparison images and the representative three-dimensional object models. 
14. The image matching system according to claim 3, comprising: 
image input means for inputting the input image! 
25 a representative three-dimensional object model storage section for 

storing a plurality of representative three-dimensional object models; 
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image generation means for generating at least one comparison 
image close in input condition to the input image every representative 
three-dimensional object model on the basis of the representative three- 
dimensional object models stored in the representative three-dimensional 
5 object model storage section; 

image matching means for calculating a similarity between the 
input image and each of the comparison images generated by the image 
generation means, selecting a maximum similarity with respect to 
comparison images associated with each representative three-dimensional 
10 object model, and regarding the maximum similarity as a similarity 

between the input image and the representative three-dimensional object 
model; 

a reference image storage section for storing the reference images of 

objects! 

15 a reference image matching result storage section for storing 

similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
in the representative three-dimensional object model storage section; 

result matching means for extracting the reference image similar to 

20 the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated by the image 
matching means and similarities between the reference images and the 
representative three-dimensional object models stored in the reference 
image matching result storage section; 

25 a reference three-dimensional object model storage section for 

storing reference three-dimensional object models associated with the 
reference images stored in the reference image storage section; 
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image conversion means for obtaining reference three-dimensional 
object models associated with reference images extracted from the result 
matching means, from the reference three-dimensional object model storage 
section, squaring an input condition of the input image with that of the 
5 reference image extracted by the result matching means by converting the 
input image and/or the reference image extracted by the result matching 
means, on the basis of the obtained reference three-dimensional object 
models, and generating partial images respectively of the input image and 
the reference image squared in input condition with each other; and 
1 0 partial image matching means for calculating a similarity between 

the partial image of the input image and the partial image of the reference 
image generated by the image conversion means. 

15. The image matching system according to claim 14, further 
comprising* 

1 5 three-dimensional object model registration means for registering 

representative three-dimensional object models in the representative three- 
dimensional object model storage section! 

reference image registration means for registering reference images 
in the reference image storage section; and 

20 reference image matching result update means for conducting 

calculation of the similarities using the image matching means, on a 
combination of a reference image and a representative three-dimensional 
object model newly generated by registration, when a new representative 
three-dimensional object model is registered in the representative three- 

25 dimensional object model storage section by the three-dimensional object 
model registration means, or when a reference image is registered in the 
reference image storage section by the reference image registration means, 
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and adding a result of the calculation to results in the reference image 
matching result storage section; and 

three-dimensional object model generation means responsive to 
registration of a similarity between the reference image and the 
5 representative three-dimensional object model in the reference image 

matching result storage section conducted by the reference image matching 
result update means, for generating the reference three-dimensional object 
model associated with the reference image by combining the representative 
three-dimensional object models stored in the representative three- 
10 dimensional object model storage section on the basis of the similarity, and 
registering the generated reference three-dimensional object model in the 
reference three-dimensional object model storage section. 

16. The image matching system according to claim 15, wherein 
the three-dimensional object model generation means generates a 

1 5 reference three-dimensional object model associated with each reference 
■ image by combining representative three-dimensional object models stored 
in the representative three-dimensional object model storage section every 
partial region, on the basis of similarities obtained every partial region 
between reference images stored in the reference image storage section and 

20 representative three-dimensional object models stored in the representative 
three-dimensional object model storage section, and registers the generated 
reference three-dimensional object model in the reference three-dimensional 
object model storage section. 

17. The image matching system according to claim 14, wherein 

25 the image matching means calculates a similarity between the input 

image and a representative three-dimensional object model every partial 
region, 
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the reference image matching result storage section stores 
similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
in the representative three-dimensional object model storage section, every 
partial region, and 

the result matching means extracts the reference image similar to 
the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated by the image 
matching means every partial region and similarities between the reference 
images and the representative three-dimensional object models, stored in 
the reference image matching result storage section every partial region. 

18. The image matching system according to claim 14, wherein 
the result matching means calculates similarities between 

similarities between the input image and the representative three- 
dimensional object models and similarities between the reference images 
and the representative three-dimensional object models, and in the 
calculation, provides the resultant similarities with weights on the basis of 
candidate precedence of similarities between the input image and the 
comparison images and the representative three-dimensional object models. 

19. The image matching system according to claim 1, wherein the 
object is a human face. 

20. An image matching method for retrieving a reference image 
similar to an input image, the image matching method comprising: 

a step of making a match between the input image and a plurality of 
representative three-dimensional object models! 

a step of making a match between the reference image and the 
representative three-dimensional object models! and 
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a step of retrieving the reference image similar to the input image 
by using a result of the match between the input image and the 
representative three-dimensional object models and a result of the match 
between the reference image and the representative three-dimensional 
5 object models. 

21. The image matching method according to claim 20, further 
comprising: 

a step of finding a reference three-dimensional object model 
associated with the reference image similar to the input image; and 
10 a step of newly retrieving the reference image similar to the input 

image by using the reference three-dimensional object model and the input 
image. 

22. The image matching method according to claim 20, further 
comprising the steps of 

1 5 a step of finding a reference three-dimensional object model 

associated with the reference image similar to the input image! 

a conversion step of squaring an input condition of the input image 
with that of the reference image by converting the input image and/or the 
reference image on the basis of the reference three-dimensional object 

20 model; and 

a step of retrieving the reference image associated with the input 
image by making a match between the input image and reference image 
squared in input condition. 

23. The image matching method according to claim 22, wherein at 
25 the conversion step, the reference image is previously converted, and an 

input condition of the input image is squared with that of the reference 
image. 
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24. The image matching method according to claim 20, comprising- 
an image input step of inputting the input image; 
a step of storing a plurality of representative three-dimensional 
object models in a representative three-dimensional object model storage 
5 section; 

an image generation step of generating at least one comparison 
image close in input condition to the input image every representative 
three-dimensional object model on the basis of the representative three- 
dimensional object models stored in the representative three-dimensional 

10 object model storage section; 

an image matching step of calculating a similarity between the input 
image and each of the comparison images generated by the image 
generation means, selecting a maximum similarity with respect to 
comparison images associated with each representative three-dimensional 

1 5 object model, and regarding the maximum similarity as a similarity 

between the input image and the representative three-dimensional object 
model; 

a step of storing the reference images of objects in a reference image 
storage section! 

20 a step of storing similarities between the reference images stored in 

the reference image storage section and representative three-dimensional 
object models stored in the representative three-dimensional object model 
storage section, in a reference image matching result storage section; and 
a result matching step of extracting the reference image similar to 

25 the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated by the image 
matching means and similarities between the reference images and the 
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representative three-dimensional object models stored in the reference 
image matching result storage section. 

25. The image matching method according to claim 24, further 
comprising- 

5 a three-dimensional object model registration step of registering 

representative three-dimensional object models in the representative three- 
dimensional object model storage section; 

a reference image registration step of registering reference images in 
the reference image storage section; and 

10 a reference image matching result update step of conducting 

calculation of the similarities using the image matching means, on a 
combination of a reference image and a representative three-dimensional 
object model newly generated by registration, when a new representative 
three-dimensional object model is registered in the representative three- 

15 dimensional object model storage section by the three-dimensional object 
model registration means, or when a new reference image is registered in 
the reference image storage section by the reference image registration 
means, and adding a result of the calculation to results in the reference 
image matching result storage section. 

20 26. The image matching method according to claim 24, wherein 

at the image matching step, a similarity between the input image 
and a representative three-dimensional object model is calculated every 
partial region, 

the reference image matching result storage section stores 
25 similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
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in the representative three-dimensional object model storage section, every 
partial region, and 

at the result matching step, the reference image similar to the input 
image is extracted on the basis of similarities between the input image and 
5 the representative three-dimensional object models calculated by the image 
matching means every partial region and similarities between the reference 
images and the representative three-dimensional object models, stored in 
the reference image matching result storage section every partial region. 

27. The image matching method according to claim 24, wherein 
10 at the result matching step, similarities between similarities 

between the input image and the representative three-dimensional object 
models and similarities between the reference images and the 
representative three-dimensional object models are calculated, and in the 
calculation the resultant similarities are provided with weights on the basis 
1 5 of candidate precedence of similarities between the input image and the 

comparison images and the representative three-dimensional object models. 

28. The image matching method according to claim 21, comprising- 
an image input step of inputting the input image; 

a step of storing a plurality of representative three-dimensional 
20 object models in a representative three-dimensional object model storage 
section; 

an image generation step of generating at least one comparison 
image close in input condition to the input image every representative 
three-dimensional object model on the basis of the representative three- 
25 dimensional object models stored in the representative three-dimensional 
object model storage section; 
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an image matching step of calculating a similarity between the input 
image and each of the comparison images generated at the image generation 
step, selecting a maximum similarity with respect to comparison images 
associated with each representative three-dimensional object model, and 
regarding the maximum similarity as a similarity between the input image 
and the representative three-dimensional object model; 

a step of storing the reference images of objects in a reference image 
storage section; 

a step of storing similarities between the reference images stored in 
the reference image storage section and representative three-dimensional 
object models stored in the representative three-dimensional object model 
storage section, in a reference image matching result storage section; 

a result matching step of extracting the reference image similar to 
the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated at the image 
matching step and similarities between the reference images and the 
representative three-dimensional object models stored in the reference 
image matching result storage section; 

a step of storing reference three-dimensional object models 
associated with the reference images stored in the reference image storage 
section, in a reference three-dimensional object model storage section; 

a second image generation step of obtaining reference three- 
dimensional object models associated with reference images extracted at the 
result matching step, from the reference three-dimensional object model 
storage section, and generating at least one second comparison image close 
in input condition to the input image every reference three-dimensional 
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object model on the basis of the obtained reference three-dimensional object 
models; and 

a second image matching step of calculating similarities between the 
input image and second comparison images generated at the second image 
5 generation step, selecting a maximum similarity from among second 

comparison images associated with each of the reference three-dimensional 
object models, and regarding the maximum similarity as a similarity 
between the input image and the reference three-dimensional object model. 
29. The image matching method according to claim 28, further 
1 0 comprising- 

a three-dimensional object model registration step of registering 
representative three-dimensional object models in the representative three- 
dimensional object model storage section; 

a reference image registration step of registering reference images in 
1 5 the reference image storage section; and 

a reference image matching result update step of conducting 
calculation of the similarities using the image matching means, on a 
combination of a reference image and a representative three-dimensional 
object model newly generated by registration, when a new representative 
20 three-dimensional object model is registered in the representative three- 
dimensional object model storage section at the three-dimensional object 
model registration step, or when a new reference image is registered in the 
reference image storage section at the reference image registration step, and 
adding a result of the calculation to results in the reference image matching 
25 result storage section; and 

a three-dimensional object model generation step of, in response to 
registration of a similarity between the reference image and the 
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representative three-dimensional object model in the reference image 
matching result storage section conducted at the reference image matching 
result update step, generating the reference three-dimensional object model 
associated with the reference image by combining the representative three- 

5 dimensional object models stored in the representative three-dimensional 
object model storage section on the basis of the similarity, and registering 
the generated reference three-dimensional object model in the reference 
three-dimensional object model storage section. 

30. The image matching method according to claim 29, wherein 

1 0 at the three-dimensional object model generation step, a reference 

three-dimensional object model associated with each reference image is 
generated by combining representative three-dimensional object models 
stored in the representative three-dimensional object model storage section 
every partial region, on the basis of similarities obtained every partial 

1 5 region between reference images stored in the reference image storage 
section and representative three-dimensional object models stored in the 
representative three-dimensional object model storage section, and the 
generated reference three-dimensional object model is registered in the 
reference three-dimensional object model storage section. 

20 31. The image matching method according to claim 28, wherein 

at the image matching step, a similarity between the input image 
and a representative three-dimensional object model is calculated every 
partial region, 

the reference image matching result storage section stores 
25 similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
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in the representative three-dimensional object model storage section, every 
partial region, and 

at the result matching step, the reference image similar to the input 
image is extracted on the basis of similarities between the input image and 
5 the representative three-dimensional object models calculated by the image 
matching means every partial region and similarities between the reference 
images and the representative three-dimensional object models, stored in 
the reference image matching result storage section every partial region. 

32. The image matching method according to claim 28, wherein 
10 at the result matching step, similarities between similarities 

between the input image and the representative three-dimensional object 
models and similarities between the reference images and the 
representative three-dimensional object models are calculated, and in the 
calculation, the resultant similarities are provided with weights on the basis 
1 5 of candidate precedence of similarities between the input image and the 

comparison images and the representative three-dimensional object models. 

33. The image matching method according to claim 22, comprising" 
an image input step of inputting the input image; 

a step of storing a plurality of representative three-dimensional 
20 object models in a representative three-dimensional object model storage 
section; 

an image generation step of generating at least one comparison 
image close in input condition to the input image every representative 
three-dimensional object model on the basis of the representative three- 
25 dimensional object models stored in the representative three-dimensional 
object model storage section; 
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an image matching step of calculating a similarity between the input 
image and each of the comparison images generated at the image generation 
means, selecting a maximum similarity with respect to comparison images 
associated with each representative three-dimensional object model, and 
5 regarding the maximum similarity as a similarity between the input image 
and the representative three-dimensional object model; 

a step of storing the reference images of objects in a reference image 
storage section; 

a step of storing similarities between the reference images stored in 
10 the reference image storage section and representative three-dimensional 
object models stored in the representative three-dimensional object model 
storage section, in a reference image matching result storage section; 

a result matching step of extracting the reference image similar to 
the input image on the basis of similarities between the input image and the 
15 representative three-dimensional object models calculated at the image 
matching step and similarities between the reference images and the 
representative three-dimensional object models stored in the reference 
image matching result storage section; 

a step of storing reference three-dimensional object models 
20 associated with the reference images stored in the reference image storage 
section, in a reference three-dimensional object model storage section; 

an image conversion step of obtaining reference three-dimensional 
object models associated with reference images extracted at the result 
matching step, from the reference three-dimensional object model storage 
25 section, squaring an input condition of the input image with that of the 
reference image extracted at the result matching step by converting the 
input image and/or the reference image extracted at the result matching 
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step, on the basis of the obtained reference three-dimensional object models, 
and generating partial images respectively of the input image and the 
reference image squared in input condition with each other; and 

a partial image matching step of calculating a similarity between 
5 the partial image of the input image and the partial image of the reference 
image generated at the image conversion step. 

34. The image matching method according to claim 33, further 
comprising- 

a three-dimensional object model registration step of registering 
10 representative three-dimensional object models in the representative three- 
dimensional object model storage section; 

a reference image registration step of registering reference images in 
the reference image storage section! and 

a reference image matching result update step of conducting 
1 5 calculation of the similarities at the image matching step, on a combination 
of a reference image and a representative three-dimensional object model 
newly generated by registration, when a new representative three- 
dimensional object model is registered in the representative three- 
dimensional object model storage section at the three-dimensional object 
20 model registration step, or when a reference image is registered in the 

reference image storage section at the reference image registration step, and 
adding a result of the calculation to results in the reference image matching 
result storage section; and 

a three-dimensional object model generation step of, in response to 
25 registration of a similarity between the reference image and the 

representative three-dimensional object model in the reference image 
matching result storage section conducted at the reference image matching 
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result update step, for generating the reference three-dimensional object 
model associated with the reference image by combining the representative 
three-dimensional object models stored in the representative three- 
dimensional object model storage section on the basis of the similarity, and 
5 registering the generated reference three-dimensional object model in the 
reference three-dimensional object model storage section. 

35. The image matching method according to claim 34, wherein 
at the three-dimensional object model generation step, a reference 

three-dimensional object model associated with each reference image is 
1 0 generated by combining representative three-dimensional object models 
stored in the representative three-dimensional object model storage section 
every partial region, on the basis of similarities obtained between reference 
images stored in the reference image storage section and representative 
three-dimensional object models stored in the representative three- 
1 5 dimensional object model storage section, and the generated reference, three- 
dimensional object model is registered in the reference three-dimensional 
object model storage section. 

36. The image matching method according to claim 33, wherein 
at the image matching step, a similarity between the input image 

20 and a representative three-dimensional object model is calculated every 
partial region, 

the reference image matching result storage section stores 
similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
25 in the representative three-dimensional object model storage section, every 
partial region, and 
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at the result matching step, the reference image similar to the input 
image is extracted on the basis of similarities between the input image and 
the representative three-dimensional object models calculated at the image 
matching step every partial region and similarities between the reference 
5 images and the representative three-dimensional object models, stored in 
the reference image matching result storage section every partial region. 
37. The image matching method according to claim 33, wherein 
at the result matching step, similarities between similarities 
between the input image and the representative three-dimensional object 
1 0 models and similarities^between the reference images and the . 

representative three-dimensional object models are calculated, and in the 
calculation, the resultant similarities are provided with weights on the basis 
of candidate precedence of similarities between the input image and the 
comparison images and the representative three-dimensional object models. 
1 5 38. The image matching method according to claim 20, wherein the 

object is a human face. 

39. A program for making a computer execute an image matching 
method to retrieve a reference image similar to an input image, the image 
matching method comprising: 
20 a step of making a match between the input image and a plurality of 

representative three-dimensional object models; 

a step of making a match between the reference image and the 
representative three-dimensional object models; and 

a step of retrieving the reference image similar to the input image 
25 by using a result of the match between the input image and the 

representative three-dimensional object models and a result of the match 
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between the reference image and the representative three-dimensional 
object models. 

40. The image matching program according to claim 39, the image 
matching method further comprising' 
5 a step of finding a reference three-dimensional object model 

associated with the reference image similar to the input image; and 

a step of newly retrieving the reference image similar to the input 
image by using the reference three-dimensional object model and the input 
image. 

10 41. The image matching program according to claim 39, the image 

matching method further comprising the steps of 

a step of finding a reference three-dimensional object model 
associated with the reference image similar to the input image; 

a conversion step of squaring an input condition of the input image 
1 5 with that of the reference image by converting the input image and/or the 
reference image on the basis of the reference three-dimensional object 
model; and 

a step of retrieving the reference image associated with the input 
image by making a match between the input image and reference image 
20 squared in input condition. 

42. The image matching program according to claim 41, wherein at 
the conversion step, the reference image is previously converted, and an 
input condition of the input image is squared with that of the reference 
image. 

25 43. The image matching program according to claim 39, the image 

matching method comprising: 

an image input step of inputting the input image! 
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a step of storing a plurality of representative three-dimensional 
object models in a representative three-dimensional object model storage 
section; 

an image generation step of generating at least one comparison 
5 image close in input condition to the input image every representative 
three-dimensional object model on the basis of the representative three- 
dimensional object models stored in the representative three-dimensional 
object model storage section," 

an image matching step of calculating a similarity between the input 
10 image and each of the comparison images generated by the image 
generation means, selecting a maximum similarity with respect to 
comparison images associated with each representative three-dimensional 
object model, and regarding the maximum similarity as a similarity 
between the input image and the representative three-dimensional object 
1 5 model; 

a step of storing the reference images of objects in a reference image 
storage section; 

a step of storing similarities between the reference images stored in 
the reference image storage section and representative three-dimensional 

20 object models stored in the representative three-dimensional object model 
storage section, in a reference image matching result storage section; and 
a result matching step of extracting the reference image similar to 
the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated by the image 

25 matching means and similarities between the reference images and the 
representative three-dimensional object models stored in the reference 
image matching result storage section. 
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44. The image matching program according to claim 43, the image 
matching method further comprising' 

a three-dimensional object model registration step of registering 
representative three-dimensional object models in the representative three- 
5 dimensional object model storage section! 

a reference image registration step of registering reference images in 
the reference image storage section; and 

a reference image matching result update step of conducting 
calculation of the similarities using the image matching means, on a 
10 combination of a reference image and a representative three-dimensional 
object model newly generated by registration, when a new representative 
three-dimensional object model is registered in the representative three- 
dimensional object model storage section by the three-dimensional object 
model registration means, or when a new reference image is registered in 
15 the reference image storage section by the reference image registration 
means, and adding a result of the calculation to results in the reference 
image matching result storage section. 

45. The image matching program according to claim 43, wherein 
at the image matching step, a similarity between the input image 

20 and a representative three-dimensional object model is calculated every 
partial region, 

the reference image matching result storage section stores 
similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
25 in the representative three-dimensional object model storage section, every 
partial region, and 
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at the result matching step, the reference image similar to the input 
image is extracted on the basis of similarities between the input image and 
the representative three-dimensional object models calculated by the image 
matching means every partial region and similarities between the reference 
5 images and the representative three-dimensional object models, stored in 
the reference image matching result storage section every partial region. 
46. The image matching program according to claim 43, wherein 
at the result matching step, similarities between similarities 
between the input image and the representative three-dimensional object 
10 models and similarities between the reference images and the 

representative three-dimensional object models are calculated, and in the 
calculation the resultant similarities are provided with weights on the basis 
of candidate precedence of similarities between the input image and the 
comparison images and the representative three-dimensional object models. 
1 5 47. The image matching program according to claim 40, the image 

matching method comprising: 

an image input step of inputting the input image; 
a step of storing a plurality of representative three-dimensional 
object models in a representative three-dimensional object model storage 
20 section; 

an image generation step of generating at least one comparison 
image close in input condition to the input image every representative 
three-dimensional object model on the basis of the representative three- 
dimensional object models stored in the representative three-dimensional 
25 object model storage section; 

an image matching step of calculating a similarity between the input 
image and each of the comparison images generated at the image generation 
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step, selecting a maximum similarity with respect to comparison images 
associated with each representative three-dimensional object model, and 
regarding the maximum similarity as a similarity between the input image 
and the representative three-dimensional object model; 
5 a step of storing the reference images of objects in a reference image 

storage section; 

a step of storing similarities between the reference images stored in 
the reference image storage section and representative three-dimensional 
object models stored in the representative three-dimensional object model 

10 storage section, in a reference image matching result storage section; 

a result matching step of extracting the reference image similar to 
the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated at the image 
matching step and similarities between the reference images and the 

15 representative three-dimensional object models stored in the reference 
image matching result storage section! 

a step of storing reference three-dimensional object models 
associated with the reference images stored in the reference image storage 
section, in a reference three-dimensional object model storage section; 

20 a second image generation step of obtaining reference three- 

dimensional object models associated with reference images extracted at the 
result matching step, from the reference three-dimensional object model 
storage section, and generating at least one second comparison image close 
in input condition to the input image every reference three-dimensional 

25 object model on the basis of the obtained reference three-dimensional object 
models! and 
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a second image matching step of calculating similarities between the 
input image and second comparison images generated at the second image 
generation step, selecting a maximum similarity from among second 
comparison images associated with each of the reference three-dimensional 
5 object models, and regarding the maximum similarity as a similarity 

between the input image and the reference three-dimensional object model. 

48. The image matching program according to claim 47, the image 
matching method further comprising- 

a three-dimensional object model registration step of registering 
10 representative three-dimensional object models in the representative three- 
dimensional object model storage section; 

a reference image registration step of registering reference images in 
the reference image storage section; and 

a reference image matching result update step of conducting 
1 5 calculation of the similarities using the image matching means, on a 

combination of a reference image and a representative three-dimensional 
object model newly generated by registration, when a new representative 
three-dimensional object model is registered in the representative three- 
dimensional object model storage section at the three-dimensional object 
20 model registration step, or when a new reference image is registered in the 
reference image storage section at the reference image registration step, and 
adding a result of the calculation to results in the reference image matching 
result storage section; and 

a three-dimensional object model generation step of, in response to 
25 registration of a similarity between the reference image and the 

representative three-dimensional object model in the reference image 
matching result storage section conducted at the reference image matching 
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result update step, generating the reference three-dimensional object model 
associated with the reference image by combining the representative three- 
dimensional object models stored in the representative three-dimensional 
object model storage section on the basis of the similarity, and registering 
5 the generated reference three-dimensional object model in the reference 
three-dimensional object model storage section. 

49. The image matching program according to claim 48, wherein 
at the three-dimensional object model generation step, a reference 

three-dimensional object model associated with each reference image is 
1 0 generated by combining representative three-dimensional object models 
stored in the representative three-dimensional object model storage section 
every partial region, on the basis of similarities obtained every partial 
region between reference images stored in the reference image storage 
section and representative three-dimensional object models stored in the 
1 5 representative three-dimensional object model storage section, and the 
generated reference three-dimensional object model is registered in the 
reference three-dimensional object model storage section. 

50. The image matching program according to claim 47, wherein 
at the image matching step, a similarity between the input image 

20 and a representative three-dimensional object model is calculated every 
partial region, 

the reference image matching result storage section stores 
similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
25 in the representative three-dimensional object model storage section, every 
partial region, and 
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at the result matching step, the reference image similar to the input 
image is extracted on the basis of similarities between the input image and 
the representative three-dimensional object models calculated by the image 
matching means every partial region and similarities between the reference 
5 images and the representative three-dimensional object models, stored in 
the reference image matching result storage section every partial region. 

51. The image matching program according to claim 47, wherein 

at the result matching step, similarities between similarities 
between the input image and the representative three-dimensional object 
10 models and similarities between the reference images and the 

representative three-dimensional object models are calculated, and in the 
calculation, the resultant similarities are provided with weights on the basis 
of candidate precedence of similarities between the input image and the 
comparison images and the representative three-dimensional object models. 
1 5 52. The image matching program according to claim 41, the image 

matching method comprising: 

an image input step of inputting the input image; 

a step of storing a plurality of representative three-dimensional 
object models in a representative three-dimensional object model storage 
20 section! 

an image generation step of generating at least one comparison 
image close in input condition to the input image every representative 
three-dimensional object model on the basis of the representative three- 
dimensional object models stored in the representative three-dimensional 
25 object model storage section; 

an image matching step of calculating a similarity between the input 
image and each of the comparison images generated at the image generation 



83 



step, selecting a maximum similarity with respect to comparison images 
associated with each representative three-dimensional object model, and 
regarding the maximum similarity as a similarity between the input image 
and the representative three-dimensional object model; 
5 a step of storing the reference images of objects in a reference image 

storage section; 

a step of storing similarities between the reference images stored in 
the reference image storage section and representative three-dimensional 
object models stored in the representative three-dimensional object model 

10 storage section, in a reference image matching result storage section; 

a result matching step of extracting the reference image similar to 
the input image on the basis of similarities between the input image and the 
representative three-dimensional object models calculated at the image 
matching step and similarities between the reference images and the 

15 representative three-dimensional object models stored in the reference 
image matching result storage section; 

a step of storing reference three-dimensional object models 
associated with the reference images stored in the reference image storage 
section, in a reference three-dimensional object model storage section; 

20 an image conversion step of obtaining reference three-dimensional 

object models associated with reference images extracted at the result 
matching step, from the reference three-dimensional object model storage 
section, squaring an input condition of the input image with that of the 
reference image extracted at the result matching step by converting the 

25 input image and/or the reference image extracted at the result matching 

step, on the basis of the obtained reference three-dimensional object models, 



84 



and generating partial images respectively of the input image and the 
reference image squared in input condition with each other; and 

a partial image matching step of calculating a similarity between 
the partial image of the input image and the partial image of the reference 
5 image generated at the image conversion step. 

53. The image matching program according to claim 52, the image 
matching method further comprising: 

a three-dimensional object model registration step of registering 
representative three-dimensional object models in the representative three- 
10 dimensional object model storage section; 

a reference image registration step of registering reference images in 
the reference image storage section; and 

a reference image matching result update step of conducting 
calculation of the similarities at the image matching step, on a combination 
15 of a reference image and a representative three-dimensional object model 
newly generated by registration, when a new representative three- 
dimensional object model is registered in the representative three- 
dimensional object model storage section at the three-dimensional object 
model registration step, or when a reference image is registered in the 
20 reference image storage section at the reference image registration step, and 
adding a result of the calculation to results in the reference image matching 
result storage section; and 

a three-dimensional object model generation step of, in response to 
registration of a similarity between the reference image and the 
25 representative three-dimensional object model in the reference image 

matching result storage section conducted at the reference image matching 
result update step, for generating the reference three-dimensional object 
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model associated with the reference image by combining the representative 
three-dimensional object models stored in the representative three- 
dimensional object model storage section on the basis of the similarity, and 
registering the generated reference three-dimensional object model in the 
5 reference three-dimensional object model storage section. 

54. The image matching program according to claim 53, wherein 
at the three-dimensional object model generation step, a reference 

three-dimensional object model associated with each reference image is 
generated by combining representative three-dimensional object models 

10 stored in the representative three-dimensional object model storage section 
every partial region, on the basis of similarities obtained every partial 
region between reference images stored in the reference image storage 
section and representative three-dimensional object models stored in the 
representative three-dimensional object model storage section, and the 

1 5 generated reference three-dimensional object model is registered in the 
reference three-dimensional object model storage section. 

55. The image matching program according to claim 52, wherein 

at the image matching means, a similarity between the input image 
and a representative three-dimensional object model is calculated every 
20 partial region, 

the reference image matching result storage section stores 
similarities between the reference images stored in the reference image 
storage section and representative three-dimensional object models stored 
in the representative three-dimensional object model storage section, every 
25 partial region, and 

at the result matching means, the reference image similar to the 
input image is extracted on the basis of similarities between the input 
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image and the representative three-dimensional object models calculated at 
the image matching step every partial region and similarities between the 
reference images and the representative three-dimensional object models, 
stored in the reference image matching result storage section every partial 
region. 

56. The image matching program according to claim 52, wherein 
at the result matching step, similarities between similarities 

between the input image and the representative three-dimensional object 
models and similarities between the reference images and the 
representative three-dimensional object models are calculated, and in the 
calculation, the resultant similarities are provided with weights on the basis 
of candidate precedence of similarities between the input image and the 
comparison images and the representative three-dimensional object models. 

57. The image matching program according to claim 39, wherein the 
object is a human face. 
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